Methods, systems, articles of manufacture and apparatus to calibrate payload information

ABSTRACT

Methods, apparatus, systems, and articles of manufacture to calibrate payload information are disclosed. An example apparatus includes at least one memory, instructions in the apparatus, and processor circuitry. The processor circuitry is to execute the instructions to obtain (a) passive measurement data and (b) background active measurement (BAM) data from panelists in a market of interest, the BAM data associated with network usage metrics of panelist wireless devices undisturbed by panelist behavior. The processor circuitry is also to execute the instructions to assign a first share of the passive measurement data and a second share of the BAM data to particular areas within the market of interest, the first share based on a number of passive measurement occurrences in a first one of the particular areas, the second share based on a number of BAM occurrences in the first one of the particular areas. The processor circuitry is also to execute the instructions to remove a bias between the passive measurement data and the BAM data by calibrating the BAM data for the market of interest using weights determined for the first one of the particular areas based on the first and second shares.

RELATED APPLICATIONS

This application arises from a continuation of U.S. patent applicationSer. No. 15/966,991, filed Apr. 30, 2018, which claims priority to U.S.Provisional Patent Application No. 62/611,797, filed Dec. 29, 2017. Theentireties of U.S. patent application Ser. No. 15/966,991 and U.S.Provisional Patent Application No. 62/611,797 are incorporated byreference herein.

FIELD OF THE DISCLOSURE

This disclosure relates generally to wireless network performance, and,more particularly, to methods, systems, articles of manufacture, andapparatus to calibrate payload information.

BACKGROUND

In recent years, wireless network providers have invested capital intoinfrastructure improvements to satisfy consumers of wireless services.Wireless network providers seek information related to which portions orlocations of their infrastructure are candidates for capital investment,thereby avoiding improvements to portions or locations that may not benecessary. Wireless network providers also wish to compare theperformance of their wireless network to competing providers. Forexample, wireless network providers seek to determine locations in whichthey are ahead of competitors and in which locations they lag behindtheir competitors. For these reasons, it is important for wirelessnetwork providers to accurately measure performance throughout differentareas and markets.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a map illustrating example passively collected data usage fora market of interest.

FIG. 1B is a map illustrating example background data usage for themarket of interest shown in FIG. 1A.

FIG. 2 is a schematic illustration of an example data calibration systemconstructed in accordance with the teachings of this disclosure toimplement the examples disclosed herein.

FIG. 3 is a flowchart representative of machine readable instructionswhich may be executed to implement the example data calibrator of FIG.2.

FIG. 4 is a flowchart representative of machine readable instructionswhich may be executed to implement the example weight generator of FIG.2.

FIG. 5 is a flowchart representative of machine readable instructionswhich may be executed to implement the example share determiner of FIG.2.

FIG. 6 is an example set of scorecards 600 created based on thecalibrated BAM data generated by the data calibrator 201 of FIG. 2.

FIG. 7 is a block diagram of an example processing platform structuredto execute the instructions of FIGS. 3-5 to implement the example datacalibrator 201.

The figures are not to scale. Instead, the thickness of the layers orregions may be enlarged in the drawings. In general, the same referencenumbers will be used throughout the drawing(s) and accompanying writtendescription to refer to the same or like parts.

DETAILED DESCRIPTION

Performance of a wireless network may fluctuate based on severaldifferent infrastructure-related influences, such as network elementage, network element technologies, network element broadcast power, basestation(s) antenna configuration(s), base station(s) antennatechnologies, etc. Additionally, the performance of a wireless networkmay fluctuate based on several operating characteristics, such asweather conditions, number of active users of the network, distributionof the active users at different portions of the network, and/or otheroperating characteristics.

To determine whether one or more markets or locations of the wirelessnetwork are candidate locations for infrastructure capital investment,network testing is performed. Two example methods of the technical fieldof network testing are employed: passive measurement testing andbackground active measurement (BAM) testing. Passive measurement testingincludes measurements of how consumers actually experience and use theirwireless network on a day-to-day basis. BAM testing includes measuringwireless network capability through controlled, regularly scheduledtesting.

However, in the technical field of network testing and market research,to isolate issues with one or more markets or locations of the wirelessnetwork that may be performing above or below one or more thresholdcriteria (e.g., bit-error-rate threshold values, transmission bandwidththreshold values, receiver bandwidth threshold values, power levelthreshold values, etc.), variability during the testing must be reducedand/or minimized. For example, in the event that a bandwidth test isperformed by one or more on-device-meters (ODMs) during a time at whichnetwork demand is relatively high, the results of the bandwidth test maybe influenced by factors not associated with infrastructure-relatedinfluences. In other words, a relatively poor result of the bandwidthtest may be caused by the relatively high network demand coincident withthe test, thereby incorrectly indicating that one or more networkelements of the test-area of interest are faulty.

To remove, reduce, and/or minimize network testing uncertainty, examplesdisclosed herein apply the BAM tests that occur during regularlyscheduled dates and/or times. In some examples, BAM tests occur onceevery ten (10) days. The BAM tests include, for example, transferring a60-second video from a test management server to a participatingpanelist wireless device. During the example BAM test, a correspondingODM on the participating panelist wireless device measures an amount oftime required to download the 60-second video, an amount of time beforethe 60-second video begins rendering of the transmitted video content, anumber of video playback stall instances, and/or other measurablecharacteristics. In some examples, video playback stall instances areindicative of network connectivity problems, bandwidth managementproblems, and/or transmitter power problems. Such problems are relatedto the infrastructure of the wireless network in an area and aretherefore important to wireless network providers engaged in thetechnical field of market research and network testing.

Additionally, examples disclosed herein ensure that user activities onthe participating panelist wireless device do not influence test resultsfrom one or more BAM tests by confirming that the participating panelistwireless device meets specific criteria prior to initiating the BAMtest(s). In other words, BAM testing is associated with network usagemetrics of panelist wireless devices that are undisturbed by panelistbehavior. For example, when the user is using the wireless device forone or more other functions (e.g., video viewing, picture viewing, webbrowsing, conducting a phone call, taking pictures, playing music,etc.), such other functions performed and/or otherwise executed by thewireless device may affect measurements related to the amount of timebefore the video begins rendering, the amount of time to download thevideo, the number of video playback stall instances, and/or other suchmeasurable characteristics (e.g., due to the other functions consumingprocessing resources of the wireless device). Additionally oralternatively, the BAM test requires that the wireless device screen isoff, the wireless device has not been turned on in the prior 5-minutes,and/or the wireless device has not moved (e.g., by monitoring on-devicemotion sensors, accelerometers, etc.) in the prior 5-minutes. In someexamples, the BAM test occurs during a time of day in which it is lesslikely that user activities will occur on the wireless device, such ashours of the day in which sleeping typically occurs.

While the aforementioned requirements before and/or during BAM testprocedures allows a greater degree of measurement accuracy that isrelated to infrastructure-related circumstances and less influenced byoperating characteristics (e.g., user interaction), such requirementscause biasing of collected data. Stated differently, because BAM testingis triggered at regular intervals rather than at instances when apanelist is actually using their wireless device and/or the wirelessnetwork, BAM test results do not necessarily reflect real networkactivity in a given market of interest. Accordingly, examples disclosedherein improve the technical field of network testing and marketresearch by weighting the BAM test results in a manner that isassociated with the passively collected data usage, thereby reducingmeasurement error and/or otherwise improving the accuracy of measuringnetwork activity within a given market or location.

In examples disclosed herein, a market of interest is selected thatincludes panelists within a same geographic area. As used herein, theterm “market of interest” is a geographic area. In some examples, themarket of interest is a metropolitan area of a major city (e.g., LosAngeles, Chicago, New York, etc.). In some examples, the market ofinterest may be one of the top 44 markets in the United States (e.g.,the top 44 markets including the greatest number of panelists). Passivemeasurement data associated with panelist network usage metrics andbackground active measurement (BAM) data associated with network usagemetrics of panelist wireless devices undisturbed by panelist behaviorare obtained from panelists. A share of the passive measurement data anda share of the BAM data are assigned to particular areas within themarket of interest, and weights for each of the particular areas aredetermined based on the share of passive measurement data and the shareof BAM data in each of the particular areas. The BAM data is thencalculated for the market of interest using the weights.

In some examples disclosed herein, the particular areas are zip codeswithin the market of interest. In some examples, the share of passivedata is determined based on a number of passive data events occurring ineach particular area and the share of active data is determined based ona number of active data events occurring in each particular area.

Some examples disclosed herein include determining the share of passivedata by calculating a mean and corresponding standard error of thenumber of passive data events within the particular areas of the marketof interest and determining the share of passive data for each zip codebased on a comparison between the number of passive data eventsoccurring in the particular area and the mean of passive events for themarket of interest. In some such examples, the share of passive data forthe particular area is the mean of the passive data events when thenumber of passive data events occurring in a particular area is within afirst threshold value above or below the mean. In some other examples,the particular areas that include a number of passive data events withina second threshold value of zero are pooled together to determine ashare of passive data for the pooled areas.

As used herein, the term “data event” refers to a transfer of data to orfrom a device (e.g., streaming audio/video, downloading app data (e.g.,weather data, etc.), etc.) and is also referred to as a “measurementoccurrence.” As used herein, the term “passive data event” refers to anypanelist activity on a device including a transfer of data to or fromthe device. Additionally, the term “active data event” as used hereinrefers to a transfer of a video (e.g., a video of a known duration,known frame rate, known file size, etc.) to a panelist device used forBAM testing. Further, as defined herein, a “passive share” refers to thenumber of passive data events occurring in a given particular area andis additionally referred to as a “share of passive data” and/or a “shareof passive measurement data.” As defined herein, an “active share”refers to the number of active data events occurring in a givenparticular area and is additionally referred to as a “share of BAMdata.”

FIG. 1A is a map 100 of passively collected data usage for an examplemarket of interest (e.g., Dallas, Tex.). In some examples, the market ofinterest is a metropolitan area of a major city (e.g., Los Angeles,Chicago, New York, etc.). In some examples, the market of interest maybe one of the top 44 markets in the United States (e.g., the top 44markets including the greatest number of panelists). In the illustratedexample of FIG. 1A, different shaded regions illustrate a usagepercentage value of a wireless network that occurs during passivemeasurements. As used herein, “passive measurements” refer to ODMS onpanelist devices to measure how consumers actually experience and usetheir wireless network on a day-to-day basis. For example, when apanelist watches a video on their mobile device, the ODM collects howmuch data is transferred to and from the device, a speed with which thedata is transferred, which application(s) are used, current networktechnology, device type, etc. Stated differently, passive measurementdata is indicative of network usage of wireless/mobile devices caused bypanelist interaction(s) with their associated mobile devices. An examplefirst region 106A of FIG. 1A indicates approximately 9% of a populationis using the wireless network, while an example second region 108Aindicates approximately 2% of a population is using the wirelessnetwork.

FIG. 1B is a map 104 of BAM testing for the same example market ofinterest (e.g., Dallas, Tex.). In the illustrated example of FIG. 1B,different shaded regions (also referred to herein as a “particulararea”) illustrate a usage percentage value of a wireless network thatoccurs during BAM testing, as described above. An example first region106B of FIG. 1B indicates a relatively lower population using thewireless network (approximately 1-2%) as compared to the example firstregion 106A of FIG. 1A, and an example second region 108B of FIG. 1Bindicates a relatively higher population using the wireless network(approximately 8-9%) as compared to the example second region 108A ofFIG. 1A.

The different usage patterns illustrated in FIGS. 1A and 1B identify abias that occurs with BAM testing. For example, consider the examplefirst region 106A and 106B of the market of interest is associated witha business district (e.g., a financial district, a manufacturingdistrict, a business/office park, etc.) that is typically populated withworkers during the day. Also consider the example second region 108A and108B of the market of interest is associated with a residential area.With that, passive measurements illustrate a relatively higher usage ofthe wireless network in the business district (e.g., the first region106A) in a manner consistent with expected usage patterns of users.However, the BAM test results associated with the example second region108B of FIG. 1B indicate a relatively high amount of network usage forthe residential area, but that indication is over-inflated due to theregularly scheduled test times (e.g., 2:00 AM when most users are likelysleeping). As such, examples disclosed herein weight BAM test results ina manner that adheres to expected usage patterns guided by the examplepassive measurement data.

FIG. 2 is a schematic illustration of an example data calibration system200 constructed in accordance with the teachings of this disclosure toimplement the examples disclosed herein. The example calibration system200 includes an example data calibrator 201. The example data calibrator201 of FIG. 2 removes bias inherent to background active measurement(BAM) data by applying weights based on passive measurement data (alsoreferred to herein as “passive data”). The example data calibrator 201of FIG. 2 is communicatively connected to an example network 202. Theexample network 202 of FIG. 2 may be the Internet, an intra-net, awide-area network (WAN), or any other network. The example network 202is communicatively connected to an example passive measurement datastore 204 and an example background active measurement (BAM) data store206. As described above, passive data stored in the example passivemeasurement data store 204 is acquired from ODMs operating and/orotherwise executing on panelist devices, such as wireless devices. Insome examples, the passive measurement data is derived and/or otherwiseacquired from the Nielsen® Mobile Performance (NMP)® platform, whichuses a smartphone meter application (e.g., an ODM) to collect a streamof data on wireless network performance and/or consumer behavior fromdevices of approximately seventy thousand panelists across the UnitedStates. In some examples, the NMP® platform collects BAM data storedwithin the BAM data store 206.

In some examples, the BAM data stored within the BAM data store 206 maybe acquired by any of the methods described above, but not limitedthereto. The acquired BAM data stored in the BAM data store 206 may bebiased due to the requirements of the data collection methods (e.g., thedevice screen must be off, the wireless device has not been turned on inthe prior 5-minutes, the wireless device has not moved in the previous5-minutes, etc.). To mitigate or eliminate the biases present in the BAMdata, the data calibrator 201 of FIG. 2 applies a weighting factor orweight to the BAM data based on the passive data.

The example data calibrator 201 of FIG. 2 includes an example dataacquirer 208. The example data acquirer 208 acquires the passive datafrom the example passive measurement data store 204 and the example BAMdata from the example BAM data store 206 via the example network 202. Insome examples, the data acquirer 208 may acquire the passive data andthe BAM data at particular intervals of time (e.g., periodic, aperiodic,scheduled, etc.), in response to a request, and/or via any other methodof data collection. In the illustrated example of FIG. 2, the dataacquirer 208 selects a market of interest and acquires the passive dataand the BAM data associated with the selected market of interest. Insome examples, a user selects data for the data acquirer 208 to acquire.In some examples, the market of interest is a metropolitan area of amajor city (e.g., Los Angeles, Chicago, New York, etc.). In someexamples, the market of interest may be one of the top 44 markets in theUnited States (e.g., the top 44 markets including the greatest number ofpanelists). The data acquirer 208 further acquires passive data and BAMdata associated with particular areas (e.g., region 106A of FIG. 1A) inthe market of interest. The particular areas discussed herein are zipcodes. However, other particular areas may also be included (e.g., celltower radius locations, counties, etc.).

In the illustrated example of FIG. 2, the data calibrator 201 includes adata cleaner 210 to clean and filter the passive data and BAM dataacquired by the data acquirer 208 for the selected market. In someexamples, the data cleaner 210 receives the passive data and the BAMdata from the data acquirer 208 via a bus 212. For example, the datacleaner 210 may discard any data events occurring prior to a specifictime period (e.g., one month, one year, etc.). The data cleaner 210 mayalso ensure the data acquired includes at least a set number ofpanelists for each provider (e.g., at least two panelists for a givenprovider).

The example data calibrator 201 of FIG. 2 further includes an exampleweight generator 214 to calculate a weighting factor or weight to beapplied to the BAM data. When the example weight generator 214 hascalculated the weight, an example BAM calibrator 216 applies the weightto the BAM data acquired by the example data acquirer 208. Theapplication of the weight eliminates or reduces the biases present inthe BAM data, thus producing more accurate BAM data to be provided tocellular network providers.

The example data calibrator 201 of FIG. 2 further includes an exampleprovider data generator 218 to generate data for use by a cellularnetwork provider based on the calibrated BAM data. For example, theprovider data generator 218 may receive the calibrated BAM data from theBAM calibrator 216 via the bus 212. The example provider data generator218 calculates metrics indicating network performance within the marketof interest for use by a provider. In some examples, the metrics includeaverage video start time, video start success rate, and/or other metricsthat may be desired by a network provider. In some examples, the metricsgenerated by the provider data generator 218 are displayed in ascorecard or other visual representation, such as the scorecarddescribed in further detail in connection with FIG. 6.

The example weight generator 214 of FIG. 2 includes a share determiner220 to determine a share of passive data to be assigned to each zip codewithin a market of interest. In other examples, a different particulararea may be used in place of the zip code. The example weight generator214 further includes an example market merger 222 to merge the passivedata and the BAM data associated with the market of interest. The marketmerger 222 is further to merge the passive data and the BAM data of eachzip code within the market of interest.

In the illustrated example of FIG. 2, the weight generator 214 furtherincludes an example weight calculator 224 to calculate the weight(s)associated with each particular area within the market of interest. Theweight calculator 224 calculates the weight(s) based on, in part, theassigned shares of the BAM data and the passive data. In some examples,the weight is calculated by a ratio of the passive data and the BAMdata. In some examples, the weight may be calculated in a mannerconsistent with example equation 1:

$\begin{matrix}{{weight} = \frac{\left( {{passive}\mspace{14mu}{{share}/{market}}\mspace{14mu}{total}\mspace{14mu}{passive}\mspace{14mu}{share}} \right)}{\left( {{active}\mspace{14mu}{{share}/{market}}{\mspace{11mu}\;}{total}\mspace{14mu}{active}\mspace{14mu}{share}} \right)}} & {{Equation}\mspace{20mu} 1}\end{matrix}$

In the illustrated example of Equation 1, the “passive share” representsthe number of passive data events occurring in a given zip code asdetermined by the share determiner 220, and the “market total passiveshare” represents the total number of passive data events occurringwithin the market of interest. Additionally, the “active share”represents the number of active data events occurring in a given zipcode as determined by the share determiner 220, and the “market totalactive share” represents the total number of active data eventsoccurring within the market of interest. Equation 1 utilizes the passiveshare and the active share assigned to each zip code in relation to thetotal share of the market for both the BAM data and the passive data.

The example weight generator 214 of FIG. 2 further includes an exampleweight adjuster 226 to adjust and/or verify the weight calculated by theweight calculator 224. In some examples, the weight adjuster 226normalizes the weights to have a mean of one (e.g., by dividing eachweight by a mean weight value). When the weight adjuster 226 hasverified the weight, the weight may be used by the BAM calibrator 216 tocalibrate the BAM data and remove the bias present in the BAM data.

To determine the share of the passive data assigned to each zip code,the example share determiner 220 of FIG. 2 includes an example marketshare analyzer 228, an example zip code selector 230, an example sharecomparator 232, an example share assignor 234, and an example zip codecombiner 236.

The example market share analyzer 228 of FIG. 2 estimates a share ofpassive data events in each zip code as a percentage of the total numberof passive data events occurring in the market of interest (e.g.,Dallas, Tex.). When the share of passive data events has been estimated,the example market share analyzer 228 further calculates an estimatedmean passive share of all of the zip codes in the selected market ofinterest. The example market share analyzer 228 additionally estimates astandard error of the passive shares of the zip codes in the selectedmarket of interest.

In the illustrated example of FIG. 2, the zip code selector 230 of FIG.2 selects a first zip code in the selected market of interest (e.g.,Dallas, Tex.) to be assigned a passive share. The zip code selector 230continues to select zip codes until each zip code within the market ofinterest has been selected and assigned a respective passive share. Inother examples, the zip code selector 230 may instead select differentparticular areas within the market of interest (e.g., one or morecellular tower radius locations, counties, etc.).

The example share comparator 232 of FIG. 2 compares the passive share ofthe zip code selected by the example zip code selector 230 to thepassive share estimated by the example market share analyzer 228. Theexample share comparator 232 outputs the result of the comparison to anexample share assignor 234. The example share assignor 234 assigns apassive share to the zip code. In assigning the passive share to the zipcode, the example share assignor 234 begins with the null hypothesis: anassumption that every zip code has a mean level of user activity and,therefore, should be assigned the mean weight. If a passive share of azip code is within a particular value of the mean (e.g., a thresholdvalue), the zip code is assumed to have the mean level of user activity(e.g., the zip code supports the null hypothesis) and is accordinglyassigned the mean share. On the other hand, when the passive share of azip code deviates from the mean share by a certain amount (e.g., morethan a threshold value above or below the mean), the null hypothesis isviolated, and the zip code must be weighted according to the passiveshare determined by the example market share analyzer 228.

When the example share comparator 232 determines that the passive shareof the selected zip code satisfies (e.g., is greater than) a firstthreshold value above or below the mean passive share, the example shareassignor 234 assigns the selected zip code its passive share (e.g., thepercentage of data events occurring in the selected zip code). In someexamples, the threshold value is one standard error (e.g., the standarderror estimated by the market share analyzer 228). Additionally oralternatively, the first threshold value may be larger or smaller thanone standard error. For example, if the estimated mean share is 7%(e.g., the average zip code within the market of interest includes 7% ofthe passive data events occurring within the market of interest) and thefirst threshold value (e.g., one standard error) is 1%, a passive sharebelow 6% (e.g., one standard error below the mean) and above 8% (e.g.,one standard error above the mean) would be considered more than thefirst threshold value above or below the mean passive share. Therefore,a zip code with a share of 9% will be assigned its market share (9%)because it is more than the first threshold value above the mean share,and a zip code having a passive share of 5% would be assigned itspassive share (5%) because it is more than the first threshold valuebelow the mean.

If, on the other hand, the passive share of the selected zip code iswithin the first threshold value above or below the mean, the shareassignor 234 assigns the estimated mean passive share of market ofinterest to the selected zip code. In some examples, the first thresholdvalue is one standard error (e.g., the standard error estimated by themarket share analyzer 228). Additionally or alternatively, the firstthreshold value may be larger or smaller than one standard error. Forexample, taking the estimated mean share to be 7% and the firstthreshold value to be 1%, as above, a passive share between 6% (e.g.,one standard error below the mean) and 8% (e.g., one standard errorabove the mean) would be considered less than the first threshold valueabove or below the mean passive share. Therefore, an example zip codehaving a share of 7.5% would be assigned the estimated mean share (7%)by the share assignor 234 because it falls within the first thresholdvalue (e.g., 1%) of the mean passive share.

In some examples, the share comparator 232 determines that the passiveshare of the selected zip code is within a second threshold value ofzero percent of the total market passive share (e.g., the total numberof passive data events occurring in the market of interest). Forexample, if the second threshold value is 1% of the total market passiveshare, a selected zip code having a passive share of 0.05% is between 0%and 1% (e.g., the second threshold value) and is therefore determined tobe within the second threshold value of zero percent of the total marketpassive share by the share comparator 232. In some examples, the secondthreshold value may be one standard error (e.g., the standard errorestimated by the market share analyzer 228). Additionally oralternatively, the second threshold value may be larger or smaller thanone standard error. In such an example, the share assignor 234 willassign the zip code to a pool of zip codes. The example share assignor234 adds all of the zip codes in the selected market of interest withinthe second threshold value of zero percent of the total market share tothe pool of zip codes to be combined by an example zip code combiner236. The example zip code combiner 236 combines (e.g., by adding thepassive shares) the zip code passive shares of each zip code selected bythe example zip code selector 230 having a passive share within a secondthreshold value of zero percent of the total market share. The zip codesmeeting this criteria (e.g., zip codes having a passive share within thesecond threshold value of zero percent of the total market share) arepooled together, for example, because they do not include enough dataevents alone to accurately determine a weight (e.g., the zip codes donot include enough data to be statistically significant).

The example zip code combiner 236 combines the passive shares of eachrespective zip code in the pool of zip codes. The pool of zip codes isthen treated as a single entity and assigned a passive share by theexample share assignor 234. In some examples, the share assignor 234uses the same criteria to determine the passive share to be assigned tothe pool of zip codes as was used on individual zip codes (e.g., acomparison to the first threshold value). For example, the pool of zipcodes is assigned its passive share when the share comparator 232determines that the passive share of the pool of zip codes is more thanthe first threshold above or below the mean passive share determined bythe market share analyzer 228. In such an example, if the passive shareof the pool of zip codes is within the first threshold value above orbelow the mean, the pool of zip codes is assigned the mean passive shareby the share assignor 234. In some other examples, the share assignor234 assigns the pool of zip codes its passive share (e.g., the combinedpassive share from the zip code combiner 236) without comparing the poolof zip codes to the mean passive share and the first threshold value.

The example share assignor 234 of FIG. 2 additionally determines a shareof BAM data to be assigned to each zip code within a market of interest.In some examples, the share assignor 234 assigns a share of the BAM datato each zip code based on a number of active data events that occurwithin the zip code. In some such examples, the assigned share may be apercentage of the active data events in the market of interest thatoccur in each zip code.

When the example share assignor 234 of FIG. 2 has assigned a passiveshare to each zip code in the market of interest, the passive shares canbe used by the example market merger 222 to combine with the BAM sharesassigned by the example share assignor 234. The passive shares and theBAM shares are then used to calculate the weights for each zip code bythe example weight calculator 224 and applied to the BAM data by theexample BAM calibrator 216 to eliminate or reduce the biases in the BAMdata, as described above.

In the illustrated example of FIG. 2, the data acquirer 208 implements ameans for acquiring, the share assignor 234 implements a means forassigning, the weight calculator 224 implements a means for calculating,the BAM calibrator 216 implements a means for calibrating, the marketshare analyzer 228 implements a means for analyzing, the zip codecombiner 236 implements a means for combining, the data cleaner 210implements a means for cleaning, the provider data generator 218implements a means for generating, the market merger 222 implements ameans for merging, the weight adjuster 226 implements a means foradjusting, the zip code selector 230 implements a means for selecting,and the share comparator 232 implements a means for comparing.

While an example manner of implementing the data calibrator 201 isillustrated in FIG. 2, one or more of the elements, processes and/ordevices illustrated in FIG. 2 may be combined, divided, re-arranged,omitted, eliminated and/or implemented in any other way. Further, theexample data acquirer 208, the example data cleaner 210, the exampleweight generator 214, the example BAM calibrator 216, the examplesprovider data generator 218, the example share determiner 220, theexample market merger 222, the example weight calculator 224, theexample weight adjuster 226, the example market share analyzer 228, theexample zip code selector 230, the example share comparator 232, theexample share assignor 234, and the example zip code combiner 236and/or, more generally, the example data calibrator 201 of FIG. 2 may beimplemented by hardware, software, firmware and/or any combination ofhardware, software and/or firmware. Thus, for example, any of theexample data acquirer 208, the example data cleaner 20210, the exampleweight generator 214, the example BAM calibrator 216, the examplesprovider data generator 218, the example share determiner 220, theexample market merger 222, the example weight calculator 224, theexample weight adjuster 226, the example market share analyzer 228, theexample zip code selector 230, the example share comparator 232, theexample share assignor 234, and the example zip code combiner 236,and/or, more generally, the example data calibrator 201 of FIG. 2 couldbe implemented by one or more analog or digital circuit(s), logiccircuits, programmable processor(s), programmable controller(s),graphics processing unit(s) (GPU(s)), digital signal processor(s)(DSP(s)), application specific integrated circuit(s) (ASIC(s)),programmable logic device(s) (PLD(s)) and/or field programmable logicdevice(s) (FPLD(s)). When reading any of the apparatus or system claimsof this patent to cover a purely software and/or firmwareimplementation, at least one of the example data acquirer 208, theexample data cleaner 210, the example weight generator 214, the exampleBAM calibrator 216, the example provider data generator 218, the exampleshare determiner 220, the example market merger 222, the example weightcalculator 224, the example weight adjuster 226, the example marketshare analyzer 228, the example zip code selector 230, the example sharecomparator 232, the example share assignor 234, and the example zip codecombiner 236 and/or, more generally, the example data calibrator 201 ofFIG. 2 is/are hereby expressly defined to include a non-transitorycomputer readable storage device or storage disk such as a memory, adigital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc.including the software and/or firmware. Further still, the example datacalibrator 201 of FIG. 2 may include one or more elements, processesand/or devices in addition to, or instead of, those illustrated in FIG.2, and/or may include more than one of any or all of the illustratedelements, processes and devices. As used herein, the phrase “incommunication,” including variations thereof, encompasses directcommunication and/or indirect communication through one or moreintermediary components, and does not require direct physical (e.g.,wired) communication and/or constant communication, but ratheradditionally includes selective communication at periodic intervals,scheduled intervals, aperiodic intervals, and/or one-time events.

Flowcharts representative of example hardware logic, machine readableinstructions, hardware implemented state machines, and/or anycombination thereof for implementing the data calibrator 201 of FIG. 2are shown in FIGS. 3-5. The machine readable instructions may be anexecutable program or portion of an executable program for execution bya computer processor such as the processor 712 shown in the exampleprocessor platform 700 discussed below in connection with FIG. 7. Theprogram may be embodied in software stored on a non-transitory computerreadable storage medium such as a CD-ROM, a floppy disk, a hard drive, aDVD, a Blu-ray disk, or a memory associated with the processor 712, butthe entire program and/or parts thereof could alternatively be executedby a device other than the processor 712 and/or embodied in firmware ordedicated hardware. Further, although the example program is describedwith reference to the flowcharts illustrated in FIGS. 3-5, many othermethods of implementing the example data calibrator 201 mayalternatively be used. For example, the order of execution of the blocksmay be changed, and/or some of the blocks described may be changed,eliminated, or combined. Additionally or alternatively, any or all ofthe blocks may be implemented by one or more hardware circuits (e.g.,discrete and/or integrated analog and/or digital circuitry, an FPGA, anASIC, a comparator, an operational-amplifier (op-amp), a logic circuit,etc.) structured to perform the corresponding operation withoutexecuting software or firmware.

As mentioned above, the example processes of FIGS. 3-5 may beimplemented using executable instructions (e.g., computer and/or machinereadable instructions) stored on a non-transitory computer and/ormachine readable medium such as a hard disk drive, a flash memory, aread-only memory, a compact disk, a digital versatile disk, a cache, arandom-access memory and/or any other storage device or storage disk inwhich information is stored for any duration (e.g., for extended timeperiods, permanently, for brief instances, for temporarily buffering,and/or for caching of the information). As used herein, the termnon-transitory computer readable medium is expressly defined to includeany type of computer readable storage device and/or storage disk and toexclude propagating signals and to exclude transmission media.

“Including” and “comprising” (and all forms and tenses thereof) are usedherein to be open ended terms. Thus, whenever a claim employs any formof “include” or “comprise” (e.g., comprises, includes, comprising,including, having, etc.) as a preamble or within a claim recitation ofany kind, it is to be understood that additional elements, terms, etc.may be present without falling outside the scope of the correspondingclaim or recitation. As used herein, when the phrase “at least” is usedas the transition term in, for example, a preamble of a claim, it isopen-ended in the same manner as the term “comprising” and “including”are open ended. The term “and/or” when used, for example, in a form suchas A, B, and/or C refers to any combination or subset of A, B, C such as(1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, (6) Bwith C, and (7) A with B and with C.

FIG. 3 is a flowchart representative of machine readable instructionswhich may be executed to implement the example data calibration system200 and, in particular, the example data calibrator 201 of FIG. 2. Theexample program 300 begins at block 302 where the example data acquirer208 selects a market of interest. In some examples, the market ofinterest is a metropolitan area of a major city (e.g., Los Angeles,Chicago, New York, etc.). In some examples, the market of interest maybe one of the top 44 markets in the United States (e.g., the top 44markets including the greatest number of panelists).

At block 304, the example data calibrator 201 acquires demographicallyweighted passive measurement data for the selected market of interest.In some examples, the data acquirer 208 acquires the passive measurementdata from the example passive measurement data store 204 via the examplenetwork 202 of FIG. 2. At block 306, the data calibrator 201 acquiresbackground active measurement (BAM) data for the market of interest. Insome examples, the data acquirer 208 of the data calibrator 201 acquiresthe BAM data from the example BAM data store 206 via the network 202.For example, the data acquirer 208 is communicatively coupled to thepassive measurement store 204 and the BAM store 206 via a network (e.g.,the example network 202 of FIG. 2).

At block 308, the example data calibrator 201 cleans and filters theacquired data from the acquired demographically weighted passivemeasurement data (e.g., associated with block 304) and the acquired BAMdata (e.g., associated with block 306). For example, the data cleaner210 of the data calibrator 201 of FIG. 2 cleans and filters the passivedata and the BAM data acquired by the data acquirer 208. In someexamples, the data cleaner 210 receives the passive data and the BAMdata from the data acquirer 208 via the bus 212. For example, the datacleaner 210 may discard any data events occurring prior to a specifictime period (e.g., one month, one year, etc.). The data cleaner 20210may also ensure the data acquired includes at least a set (e.g., athreshold) number of panelists for each provider (e.g., at least twopanelists for a given provider) to ensure that a quantity of data issufficient to satisfy best industry practices of statisticalsignificance.

At block 310, the data calibrator 201 calculates weights. For example,the weight generator 214 of the data calibrator 201 of FIG. 2 calculatesa weight for each zip code within the selected market of interest (e.g.,particular zip codes within the Dallas, Tex. area shown in FIGS. 1A and1B). In some examples, the weight generator 214 assigns shares ofpassive data and shares of BAM data corresponding to each zip code tocalculate the weights for each zip code. A more detailed description ofblock 310 is shown in connection with FIG. 4, discussed in furtherdetail below.

At block 312, the data calibrator 201 calibrates the BAM data with theweights calculated at block 310. For example, the example BAM calibrator216 of the data calibrator 201 of FIG. 2 applies the weights calculatedby the example weight generator 214 to the BAM data associated with eachzip code.

At block 314, the data calibrator 201 generates provider data based onthe calibrated BAM data. For example, the provider data generator 218 ofthe data calibrator 201 of FIG. 2 receives calibrated BAM data from theBAM calibrator 216 (e.g., via the example bus 212 of FIG. 2). Theexample provider data generator 218 calculates metrics indicatingnetwork performance within the market of interest for use by a provider.In some examples, the metrics include average video start time, videostart success rate, and other metrics that may be desired by a networkprovider. In some examples, the provider data generated by the providerdata generator 218 is displayed in a scorecard or other visualrepresentation, such as the scorecard described in more detail inconnection with FIG. 6. The provider data generated by the exampleprovider data generator 218 represents information with reduced bias dueto the calibration techniques disclosed herein. When the exampleprovider data generator 218 generates provider data based on thecalibrated BAM data, the program 300 of FIG. 3 concludes.

FIG. 4 is a flowchart representative of machine readable instructionswhich may be executed to implement the example weight generator 214 ofFIG. 2 to calculate weights, as discussed above in connection with block310. The example program 310 begins at block 402 where the weightgenerator 214 assigns shares of passive data for each zip code. Forexample, the share determiner 220 assigns a share of passive data toeach zip code within the market of interest. A more detailed descriptionof block 402 is shown in connection with FIG. 4, as discussed in furtherdetail below.

At block 404, the example weight generator 214 assigns a share of BAMdata for each zip code based on the acquired BAM data. For example, theshare assignor 234 of FIG. 2 assigns a share of the BAM data to each zipcode based on a number of active data events that occur within the zipcode. In some such examples, the assigned share may be a percentage ofthe active data events in the market of interest that occur in each zipcode.

At block 406, the example weight generator 214 merges the passive dataand the BAM data within the market of interest. For example, the marketmerger 222 of FIG. 2 merges the passive data and the BAM data after thepassive shares and the active shares have been assigned. At block 408,the example weight generator 214 calculates the weight of each zip codebased on the assigned shares of BAM data and passive data for each zipcode. For example, the weight calculator 224 of FIG. 2 receives theassigned shares of the passive data and the BAM data from the sharedeterminer 220 (e.g., via the bus 212) and calculates the weight basedon a ratio between the passive data and the BAM data. In some examples,the weight calculator 224 may use the following equation to calculatethe weights in a manner consistent with example Equation 1, as describedabove.

In such examples, the market total passive share is the total number ofpassive data events in a market of interest (e.g., Dallas, Tex.) and themarket total active share is the number of active data events in themarket of interest (e.g., Dallas, Tex.).

At block 410, the example weight generator 214 normalizes the weights tohave a mean equal to one. For example, the weight adjuster 226normalizes each weight calculated by the weight calculator 224 to makethe mean of the weights equal to one to make comparisons betweendifferent markets possible (e.g., because all markets of interest arenormalized to the same value (e.g., one)). In some examples, the weightadjuster 226 normalizes the weights by dividing each weight by the meanvalue of the weights in the market of interest. When the weights havebeen normalized by the example weight adjuster 226, the program 310concludes.

FIG. 5 is a flowchart representative of machine readable instructionswhich may be executed to implement the example share determiner 220 ofFIG. 2, and illustrates additional detail related to assigning a shareof passive data for each zip code (block 302 of FIG. 3). The exampleprogram 402 of FIG. 5 begins at block 502 where the example sharedeterminer 220 estimates a share of passive data events in each zip codeas a percentage of total passive data events (e.g., the totalquantity/number of passive data events occurring in the market ofinterest, such as the market of Dallas, Tex. shown in FIGS. 1A and 1B).For example, the market share analyzer 228 estimates a share of dataevents in each zip code as a percentage by dividing the number of dataevents associated with each zip code by the total number of data eventsin the market of interest.

At block 504, the example share determiner 220 estimates a mean andstandard error based on the estimated share of data events in all zipcodes. For example, the market share analyzer 228 estimates the mean ofthe estimated shares of the zip codes in the selected market ofinterest. The market share analyzer 228 further estimates the standarderror of the estimated shares of the passive data for the zip codes inthe market of interest.

At block 506, the example share determiner 220 selects a zip code withinthe market of interest. For example, the zip code selector 230 of FIG. 2selects a zip code in the market of interest to be analyzed. At block508, the share determiner 220 compares the passive share of the selectedzip code to the mean share of the market. For example, the sharecomparator 232 of FIG. 2 compares the passive share of the zip codeselected by the zip code selector 230 to the mean share of the marketestimated by the market share analyzer 228.

At block 510, the share determiner 220 determines whether the share ofthe selected zip code is more than a first threshold value above orbelow the mean share. In some examples, the first threshold value is onestandard error (e.g., the standard error estimated by the example marketshare analyzer 228). Additionally or alternatively, the first thresholdvalue may be larger or smaller than one standard error. For example, theshare assignor 234 may determine whether the passive share of theselected zip code is more than one standard error above or below themean based on the comparison from the share comparator 232. If thepassive share of the selected zip code is not more than one standarderror above or below the mean share, control of the example program 402proceeds to block 512. If the passive share of the selected zip code ismore than the first threshold value above or below the mean share,control proceeds to block 514.

At block 512, the share determiner 220 assigns the mean share of themarket to the selected zip code. For example, when the comparison fromthe share comparator 232 determines that the passive share is less thanthe first threshold value of above or below the mean share, the shareassignor 234 of FIG. 2 assigns the mean share of the market to theselected zip code. For example, if the market share analyzer 228determines that the mean passive share is 6.5% and the first thresholdvalue (e.g., the standard error) is 1.5%, the share comparator 232 willdetermine that a passive share of the selected zip code that is between5% and 8% is within the first threshold value (e.g., within one standarderror) above or below the mean. Thus, the selected zip code will beassigned the mean passive share by the example share assignor 234 (e.g.,instead of the passive share of the selected zip code).

At block 514, the share determiner 220 determines whether the passiveshare of the selected zip code is within a second threshold value ofzero percent of the total market passive share (e.g., the total numberof passive data events occurring in the market of interest). In someexamples, the second threshold value is one standard error (e.g., thestandard error estimated by the market share analyzer 228). Additionallyor alternatively, the first threshold value may be larger or smallerthan one standard error. For example, the market share analyzer 228determines that the second threshold value (e.g., the standard error) is1.5%, and the share comparator 232 determines whether the passive shareis within 1.5% of zero percent of the total market share (e.g., between0% and 1.5%). If the example share comparator 232 determines that thepassive share is not within the second threshold value of zero percentof the total market share, control of the program 402 proceeds to block516. If the example share comparator 232 determines that the passiveshare is within the second threshold value of zero percent of the totalmarket share, control proceeds to block 518.

At block 516, the example share determiner 220 assigns the zip codepassive share to the selected zip code. For example, the share assignor234 assigns the selected zip code its passive share (e.g., the passiveshare calculated by the market share analyzer 228) when the sharecomparator 232 determines that the share is more than the firstthreshold value above or below the mean passive share and that thepassive share is not within the second threshold value of zero. Forexample, the market share analyzer 228 may determine that the meanpassive share is 6.5%, the standard error is 1.5%, and the passive shareof the selected zip code is 3%. In such an example, if the firstthreshold value is set to be the standard error, the share comparator232 determines that the passive share of the selected zip code is morethan one standard error below the mean share (i.e., less than 5%) andnot within one standard error of zero percent of the total market share(e.g., above 1.5%). In such an example, the share assignor 234 assignsthe selected zip code its passive share (3%). Control of program 402then proceeds to block 520.

At block 518, the share determiner 220 adds the selected zip code to apool of zip codes. For example, if the share comparator 232 determinesthat the passive share of the zip code is within the second thresholdvalue (e.g., the standard error) of zero, the zip code combiner 236 addsthe selected zip code to a pool of zip codes. The example pool of zipcodes combines the zip codes having passive shares that include too fewdata events to be assigned a share based on the passive share. The poolof zip codes may be treated as a single zip code to assign an adequateweight to zip codes having too few passive data events.

At block 520, the example share determiner 220 determines whether allzip codes in the market of interest have been assigned a passive share.For example, the zip code selector 230 determines whether each zip codein the market of interest has been selected by the zip code selector 230and assigned a passive share by the share assignor 234. If the examplezip code selector 230 determines that all zip codes have been assigned apassive share, control of the program 402 proceeds to block 522. If theexample zip code selector 230 determines that there are remaining zipcodes in the market of interest to be assigned a passive share, controlreturns to block 506, where the example zip code selector 230 selects anew zip code within the market of interest.

At block 522, the example share determiner 220 estimates a share for thepool of zip codes. For example, the zip code combiner 236 combines theshares of each respective zip code in the pool of zip codes. The pool ofzip codes is then treated as a single entity and assigned a single shareby the example share assignor 234. In some examples, the example shareassignor 234 uses the same criteria to determine the passive share to beassigned to the pool of zip codes as was used for individual zip codes.For example, the pool of zip codes is assigned its passive share whenthe share comparator 232 determines that the collective passive share ismore than the first threshold value above or below the mean passiveshare determined by the market share analyzer 228. In such an example,if the collective passive share of the pool of zip codes is within thefirst threshold value of the mean, the pool of zip codes is assigned themean passive share by the share assignor 234. In some other examples,the share assignor 234 assigns the pool of zip codes its collectivepassive share without comparing the pool of zip codes to the meanpassive share and the first threshold value (e.g., the standard error).

FIG. 6 is an example set of scorecards 600 created based on thecalibrated BAM data generated by the data calibrator 201 of FIG. 2,illustrating the effect of weighting the BAM data by utilizing themethods discussed in detail above. For example, the BAM calibrator 216of FIG. 2 may calibrate background active measurement (BAM) data asdescribed in connection with FIGS. 2 and 3. Based on the resultingcalibrated BAM data, the example provider data generator 218 of FIG. 2may calculate provider information for one or more providers that isused to accurately describe the performance of the provider in a givenmarket of interest.

The example set of scorecards 600 of FIG. 6 includes an example weightedscorecard 602 and an example unweighted provider scorecard 604. Theunweighted scorecard 604 is not provided to the network providers usingthe methods disclosed herein and is displayed in FIG. 6 for illustrativepurposes only. However, the unweighted results would have been providedto the network providers in previous methods, resulting in less accurateperformance metrics that were calculated using the BAM data prior tocalibration (e.g., before the example data calibrator 201 has removedthe bias from the BAM data). The set of scorecards 600 thereforeillustrates a comparison between the weighted scorecard 602 and theunweighted scorecard 604 to show the difference in the measurements withand without the bias.

In the illustrated example, the weighted scorecard 602 includes a marketcolumn 606 defining the market of interest (e.g., the market of interestdetermined by the example data acquirer 208 of FIG. 2). The market ofinterest in the example set of scorecards 600 is Chicago. In otherexamples, the market of interest may be any of the other markets ofinterest described in connection with FIGS. 2-5. In some examples, ascorecard such as the weighted scorecard 602 includes multiple marketsof interest (e.g., Chicago, Los Angeles, and New York, etc.). Suchinformation is useful in determining which markets of interest needadditional development (e.g., increased network infrastructure), whichmarkets are not in need of additional development, which markets ofinterest are performing most effectively, etc.

The weighted scorecard 602 further includes a metric column 608 todisplay a metric determined using the calibrated BAM data. For example,the scorecard may display information such as percent of tests with norebuffering 608A and percent of time in HD 608B (e.g., the percent oftime a sixty second video plays in high-definition). The metric column608 additionally displays metrics such as percentage of timerebuffering, average video start time, video start success rate. Inother examples, the metric column 608 may include any other metriccalculated for a provider based on the BAM data.

An example value column 610 displays the numerical values (e.g., aspercentages, decimals, integers, etc.) for each of a first provider 612and a second provider 614. For example, the weighted scorecard 602includes the metric for percent of tests with no rebuffering 608A in themetric column 608. Within the value column 610, the percentage of thetests conducted with no rebuffering is displayed as a percentage (e.g.,72.6% for the first provider 612) for the metric. A value is given foreach metric in the metric column 608 and each provider (e.g., the firstprovider 612 and the second provider 614). In some examples, theweighted scorecard 602 includes only one provider (e.g., only the firstprovider 612). In other examples, the weighted scorecard 602 includesmore than two providers.

The weighted scorecard 602 further includes a margin of error column 616to display a margin of error for each metric in the metric column 608.For example, the margin of error for the first provider 612 for thepercentage of time rebuffering metric is 1.2%. In some examples, themargin of error in the margin of error column 616 can be used todetermine a confidence interval for a given value in the value column610.

The example weighted scorecard 602 further includes a rank column 618 todetermine a rank of each provider relative to other providers. In theillustrated example, a ranking 620 of one was given to the secondprovider 614 for the percent of time rebuffering metric. The ranking 620indicates that the provider is ranked higher than competitors for thegiven metric (e.g., percent of time rebuffering). In another example, aranking of three or four would indicate that the provider is rankedlower than other competitors for the given metric. In the illustratedexample, multiple providers may receive the same ranking. For example,when the rankings are substantially similar (e.g., within a thresholdvalue of one another), the same rank may be assigned to each provider.In some alternative examples, the rank column 618 includes a uniqueranking for each provider. For example, if four providers were beingranked, each would receive a rank of one through four with no providersreceiving the same rank.

In the illustrated example of FIG. 6, the weighted scorecard 602includes a weighted entry 622 for the percent of tests with norebuffering metric 608A. The weighted entry 622 is a value displayed asa percentage. An unweighted entry 624 is shown in the unweightedscorecard 604. The weighted entry 622 and the unweighted entry 624correspond to the same market of interest (Chicago), metric (the percentof tests with no rebuffering metric 608A), and provider (the firstprovider 612). The weighted entry 622 and the unweighted entry 624illustrate the effect of the BAM data calibration, as the weighted entry622 corresponding to the weighted scorecard 602 (e.g., the calibratedBAM data) is 1.4% lower than the unweighted entry 624 corresponding tothe unweighted scorecard 604 (e.g., the uncalibrated BAM data). Thedifference between the weighted entry 622 and the unweighted entry 624represents the bias inherent in the acquired BAM data. By calibratingthe BAM data, as described in connection with FIGS. 2-5, this bias iseliminated or reduced, and the weighted scorecard 602 displays moreaccurate results than the unweighted scorecard 604. Because providersuse the BAM data to allocate spending on major infrastructure projects,for example, the accuracy and precision of the BAM data are of extremelyhigh value to providers. Inaccurate results can result in incorrectover-spending in markets where it is not needed or under-spending inmarkets where it is needed, thus greatly hindering the improvements tothe cellular networks that providers desire to make.

The example set of scorecards 600 includes other entries illustratingthe differences between the weighted scorecard 602 and the unweightedscorecard 604. Differences exist between entries in values associatedwith other metrics, values associated with either the first provider 612or the second provider 614, margin of error entries, and rankings in therank column 618. Such differences indicate the effect the bias inherentin the BAM data and the effect of weighting the BAM data using themethods disclosed herein.

FIG. 7 is a block diagram of an example processor platform 700structured to execute the instructions of FIGS. 3-5 to implement thedata calibrator 201 of FIG. 2. The processor platform 700 can be, forexample, a server, a personal computer, a workstation, a self-learningmachine (e.g., a neural network), a mobile device (e.g., a cell phone, asmart phone, a tablet such as an iPad™), a personal digital assistant(PDA), an Internet appliance, or any other type of computing device.

The processor platform 700 of the illustrated example includes aprocessor 712. The processor 712 of the illustrated example is hardware.For example, the processor 712 can be implemented by one or moreintegrated circuits, logic circuits, microprocessors, GPUs, DSPs, orcontrollers from any desired family or manufacturer. The hardwareprocessor may be a semiconductor based (e.g., silicon based) device. Inthis example, the processor implements the example data calibrator 201,the example data acquirer 208, the example data cleaner 210, the exampleweight generator 214, the example BAM calibrator 216, the example sharedeterminer 220, the example market merger 222, the example weightcalculator 224, the example weight adjuster 226, the example marketshare analyzer 228, the example zip code selector 230, the example sharecomparator 232, the example share assignor 234 and the example zip codecombiner 236 of FIG. 2.

The processor 712 of the illustrated example includes a local memory 713(e.g., a cache). The processor 712 of the illustrated example is incommunication with a main memory including a volatile memory 714 and anon-volatile memory 716 via a bus 718. The volatile memory 714 may beimplemented by Synchronous Dynamic Random Access Memory (SDRAM), DynamicRandom Access Memory (DRAM), RAMBUS® Dynamic Random Access Memory(RDRAM®) and/or any other type of random access memory device. Thenon-volatile memory 716 may be implemented by flash memory and/or anyother desired type of memory device. Access to the main memory 714, 716is controlled by a memory controller.

The processor platform 700 of the illustrated example also includes aninterface circuit 720. The interface circuit 720 may be implemented byany type of interface standard, such as an Ethernet interface, auniversal serial bus (USB), a Bluetooth® interface, a near fieldcommunication (NFC) interface, and/or a PCI express interface.

In the illustrated example, one or more input devices 722 are connectedto the interface circuit 720. The input device(s) 722 permit(s) a userto enter data and/or commands into the processor 712. The inputdevice(s) can be implemented by, for example, an audio sensor, amicrophone, a camera (still or video), a keyboard, a button, a mouse, atouchscreen, a track-pad, a trackball, isopoint and/or a voicerecognition system.

One or more output devices 724 are also connected to the interfacecircuit 720 of the illustrated example. The output devices 724 can beimplemented, for example, by display devices (e.g., a light emittingdiode (LED), an organic light emitting diode (OLED), a liquid crystaldisplay (LCD), a cathode ray tube display (CRT), an in-place switching(IPS) display, a touchscreen, etc.), a tactile output device, a printerand/or speaker. The interface circuit 720 of the illustrated example,thus, typically includes a graphics driver card, a graphics driver chipand/or a graphics driver processor.

The interface circuit 720 of the illustrated example also includes acommunication device such as a transmitter, a receiver, a transceiver, amodem, a residential gateway, a wireless access point, and/or a networkinterface to facilitate exchange of data with external machines (e.g.,computing devices of any kind) via a network 726. The communication canbe via, for example, an Ethernet connection, a digital subscriber line(DSL) connection, a telephone line connection, a coaxial cable system, asatellite system, a line-of-site wireless system, a cellular telephonesystem, etc.

The processor platform 700 of the illustrated example also includes oneor more mass storage devices 728 for storing software and/or data.Examples of such mass storage devices 728 include floppy disk drives,hard drive disks, compact disk drives, Blu-ray disk drives, redundantarray of independent disks (RAID) systems, and digital versatile disk(DVD) drives. In the illustrated example, the mass storage devices 728include the example passive measurement data store 204 and the examplebackground active measurement (BAM) data store 206.

The machine executable instructions 732 of FIGS. 2-4 may be stored inthe mass storage device 728, in the volatile memory 714, in thenon-volatile memory 716, and/or on a removable non-transitory computerreadable storage medium such as a CD or DVD.

From the foregoing, it will be appreciated that example methods,apparatus and articles of manufacture have been disclosed that calibratepayload information. In examples disclosed herein, bias associated withbackground active measurement (BAM) testing of devices in a cellularnetwork are eliminated or reduced by calibrating the BAM testing resultswith passively collected data. In some examples, the calibration of BAMdata results in more accurate measurements provided to networkproviders. Further, some examples produce information that providers mayuse to allocate spending for infrastructure in markets or locations thathave the most need of improvement. The accuracy and precision of the BAMdata are of extremely high value to providers, and the methods disclosedherein provide results that prevent incorrect over-spending in marketswhere it is not needed or under-spending in markets where it is needed.Thus, examples disclosed herein allow providers to make the improvementsto their cellular networks that provide the greatest benefit to theircustomers.

Although certain example methods, apparatus and articles of manufacturehave been disclosed herein, the scope of coverage of this patent is notlimited thereto. On the contrary, this patent covers all methods,apparatus and articles of manufacture fairly falling within the scope ofthe claims of this patent.

What is claimed is:
 1. An apparatus, comprising: at least one memory;instructions in the apparatus; and processor circuitry to execute theinstructions to: obtain (a) passive measurement data and (b) backgroundactive measurement (BAM) data from panelists in a market of interest,the BAM data associated with network usage metrics of panelist wirelessdevices undisturbed by panelist behavior; assign a first share of thepassive measurement data and a second share of the BAM data toparticular areas within the market of interest, the first share based ona number of passive measurement occurrences in a first one of theparticular areas, the second share based on a number of BAM occurrencesin the first one of the particular areas; and remove a bias between thepassive measurement data and the BAM data by calibrating the BAM datafor the market of interest using weights determined for the first one ofthe particular areas based on the first and second shares.
 2. Theapparatus of claim 1, wherein the particular areas are zip codes withinthe market of interest.
 3. The apparatus of claim 1, wherein theprocessor circuitry is to execute the instructions to: calculate a meanof the number of passive measurement occurrences within the particularareas of the market of interest; and assign the first share of thepassive measurement data to the first one of the particular areas basedon a comparison between the number of passive measurement occurrences inthe first one of the particular areas and the mean of the number ofpassive measurement occurrences.
 4. The apparatus of claim 3, whereinthe processor circuitry is to execute the instructions to assign themean of the number of passive measurement occurrences to the first oneof the particular areas when the number of passive measurementoccurrences in the first one of the particular areas is within a firstthreshold value above or below the mean.
 5. The apparatus of claim 4,wherein the first threshold value is a standard error of the number ofpassive measurement occurrences within the particular areas.
 6. Theapparatus of claim 4, wherein the processor circuitry is to execute theinstructions to combine the number of passive measurement occurrences ofthe first one of the particular areas with a number of passivemeasurement occurrences of a second one of the particular areas when thenumber of passive measurement occurrences of the first and second onesof the particular areas are within a second threshold value of zeropassive measurement occurrences.
 7. The apparatus of claim 6, whereinthe second threshold value is a standard error of the number of passivemeasurement occurrences within the particular areas.
 8. A non-transitorycomputer readable storage medium comprising instructions that, whenexecuted, cause a machine to at least: obtain (a) passive measurementdata and (b) background active measurement (BAM) data from panelists ina market of interest, the BAM data associated with network usage metricsof panelist wireless devices undisturbed by panelist behavior; assign afirst share of the passive measurement data and a second share of theBAM data to particular areas within the market of interest, the firstshare based on a number of passive measurement occurrences in a firstone of the particular areas, the second share based on a number of BAMoccurrences in the first one of the particular areas; and remove a biasbetween the passive measurement data and the BAM data by calibrating theBAM data for the market of interest using weights determined for thefirst one of the particular areas based on the first and second shares.9. The non-transitory computer readable storage medium as defined inclaim 8, wherein the particular areas are zip codes within the market ofinterest.
 10. The non-transitory computer readable storage medium asdefined in claim 8, wherein the instructions further cause the machineto: calculate a mean of the number of passive measurement occurrenceswithin the particular areas of the market of interest; and assign thefirst share of the passive measurement data to the first one of theparticular areas based on a comparison between the number of passivemeasurement occurrences in the first one of the particular areas and themean of the number of passive measurement occurrences.
 11. Thenon-transitory computer readable storage medium as defined in claim 10,wherein the instructions further cause the machine to assign the mean ofthe number of passive measurement occurrences to the first one pf theparticular areas when the number of passive measurement occurrences inthe first one of the particular areas is within a first threshold valueabove or below the mean.
 12. The non-transitory computer readablestorage medium as defined in claim 11, wherein the first threshold valueis a standard error of the number of passive measurement occurrenceswithin the particular areas.
 13. The non-transitory computer readablestorage medium as defined in claim 11, wherein the instructions furthercause the machine to combine the number of passive measurementoccurrences of the first one of the particular areas with a number ofpassive measurement occurrences of a second one of the particular areaswhen the number of passive measurement occurrences of the first andsecond ones of the particular areas are within a second threshold valueof zero passive measurement occurrences.
 14. The non-transitory computerreadable storage medium as defined in claim 13, wherein the secondthreshold value is a standard error of the number of passive measurementoccurrences within the particular areas.
 15. A method comprising:obtaining, by executing one or more instructions with processorcircuitry, (a) passive measurement data and (b) background activemeasurement (BAM) data from panelists in a market of interest, the BAMdata associated with network usage metrics of panelist wireless devicesundisturbed by panelist behavior; assigning, by executing one or moreinstructions with the processor circuitry, a first share of the passivemeasurement data and a second share of the BAM data to particular areaswithin the market of interest, the first share based on a number ofpassive measurement occurrences in a first one of the particular areas,the second share based on a number of BAM occurrences in the first oneof the particular areas; and removing, by executing one or moreinstructions with the processor circuitry, a bias between the passivemeasurement data and the BAM data by calibrating the BAM data for themarket of interest using weights determined for the first one of theparticular areas based on the first and second shares.
 16. The method asdefined in claim 15, wherein the particular areas are zip codes withinthe market of interest.
 17. The method as defined in claim 15, furtherincluding: calculating, by executing one or more instructions with theprocessor circuitry, a mean of the number of passive measurementoccurrences within the particular areas of the market of interest; andassigning, by executing one or more instructions with the processorcircuitry, the first share of the passive measurement data to the firstone of the particular areas based on a comparison between the number ofpassive measurement occurrences in the first one of the particular areasand the mean of the number of passive measurement occurrences.
 18. Themethod as defined in claim 17, further including assigning, by executingone or more instructions with the processor circuitry, the mean of thenumber of passive measurement occurrences to the first one of theparticular areas when the number of passive measurement occurrences inthe first one of the particular areas is within a first threshold valueabove or below the mean.
 19. The method as defined in claim 18, whereinthe first threshold value is a standard error of the number of passivemeasurement occurrences within the particular areas.
 20. The method asdefined in claim 18, further including combining, by executing one ormore instructions with the processor circuitry, the number of passivemeasurement occurrences of the first one of the particular areas with anumber of passive measurement occurrences of a second one of theparticular areas when the number of passive measurement occurrences ofthe first and second ones of the particular areas are within a secondthreshold value of zero passive measurement occurrences.